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Abstract 

A ssessing pragmatic knowledge in the instructed setting is seen as a 
complex but necessary task, which requires the design of appropriate 
research methodologies to examine pragmatic performance. This study 
discusses the use of two different research methodologies, namely those of 
Discourse Completion Tests/Tasks (DCTs) and verbal reports. Research has 
shown that the use of DCTs in combination with verbal reports can increase 
the trustworthiness of the results (Felix-Brasdefer, 2010). Hence, taking into 
account the potential of verbal reports, the present study aims to investigate 
the cognitive processes undertaken by a group of English language 
learners as regards their pragmatic performance. Findings regarding the 
value of retrospective verbal reports are discussed together with practical 
recommendations for the use of DCTs and verbal reports to assess speech act 
performance in the instructed setting. 
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1. Introduction 

Assessment of Second Language/Foreign Language (SL/FL) pragmatics is a 
growing area in the field of Interlanguage Pragmatics (ILP). Several researchers 
have drawn their attention to this particular aspect, especially in recent years (see 
Beltran-Palanques, 2013, 2014; Felix-Brasdefer, 2010; Roever, 2010, 2011; Ross 
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& Kasper, 2013; Uso-Juan & Martlnez-Flor, 2014). Pragmatics, which is one 
of the main components of the communicative competence model (Bachman, 
1990; Uso-Juan & Martinez-Flor, 2006), should be appropriately introduced in 
the instructed setting in order to teach and assess this competence successfully. 
In this paper, I will focus on the issue of assessing pragmatics in the online 
instructed setting combining two different research methodologies: Discourse 
Completion Tests/Tasks (DCTs) and retrospective verbal reports. In the first part 
of the paper, 1 will provide a theoretical framework which focuses on the use of 
the two aforementioned research methodologies in the field of 1LP. In the second 
part I will explain how the study was developed and the results derived from it. 
Finally, I will briefly discuss practical recommendations for the use of DCTs and 
verbal reports to assess speech act performance in the instructed setting. 


2. Theoretical framework 

Verbal reports have been used in the field of ILP in combination with other 
research instruments, particularly those of role-plays (Beltran-Palanques, 
2013; Cohen & Olshtain, 1993; Felix-Brasdefer, 2008a, 2008b; Widjaja, 
1997; Woodfield, 2012) and DCTs (Beltran-Palanques, 2013; Robinson, 1992; 
Woodfield, 2008, 2010). However, for reasons of space, this paper is restricted to 
focus on the studies in which verbal reports were employed in combination with 
DCTs (see Beltran-Palanques, 2013, 2014; Felix-Brasdefer, 2010). 

One of the pioneering studies which used verbal reports in combination with 
written DCTs was conducted by Robinson (1992). The author combined 
concurrent (single-subject think-aloud) and retrospective verbal reports (i.e. 
interviews). The data obtained by means of verbal reports provided specific 
information about the planning process of refusal semantic formulae, evaluation 
of different utterances, pragmatic and linguistic difficulties, and knowledge 
sources. Woodfield (2008) employed paired concurrent verbal reports and 
retrospective verbal reports with three pairs of native speakers of English to 
provide insights concerning issues of validity noticed during the reconstruction 
of requests in 18 written DCTs. Woodfield (2010) explored the role of paired 
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concurrent and retrospective verbal reports to examine the cognitive processes 
of advanced learners of English as a SL on written DCTs which elicited status- 
unequal requests. The data obtained from concurrent verbal reports revealed 
that the social context of the discourse situation affected the pragmalinguistic 
and sociopragmatic choices and language-related episodes showed participants’ 
negotiation of lexical and grammatical elements when planning the request 
strategies. Regarding retrospective reports, the author reported that they 
offered information about participants’ language of thought and the difficulties 
that participants experienced with the research methodology employed. More 
recently, Beltran-Palanques (2013) conducted a study employing retrospective 
verbal reports in combination with both open role-plays and interactive written 
DCTs. In this study, the speech act under investigation was that of apologies. 
Results revealed that retrospective verbal reports appeared to be instrumental in 
gathering information regarding participants’ pragmatic production. 

In short, studies using verbal reports, either concurrently or retrospectively, 
in combination with DCTs, have shown the positive effects of this particular 
research methodology. Verbal reports seem to be instrumental in providing 
information as regards the participants’ cognitive process, perceptions of 
speech act performance, validation of research instruments, sociocultural 
and sociolinguistic knowledge, as well as politeness issues. Hence, taking 
into account the literature review sketched above, the present study aims to 
contribute to this specific field of research by examining the potential of using 
interactive written DCTs in combination with retrospective verbal reports in an 
online instructed setting. 


3. The study 

3.1. Participants 

This small explanatory study included 30 adult learners (12 male and 18 female) 
of an online English as a Foreign Language (EFL) course (mean age: 28.3). 
Participants were first asked to complete the UCLES Quick Placement Test 
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(Oxford University Press) to test their proficiency level. Results revealed that 
their proficiency level was Bl, according to the Common European Framework 
of Reference for Languages (CEFR- http://www.coe. int/t/dg4/linguistic/source/ 
framework_en.pdf). This particular result was expected, as participants were taking 
an online Bl course. Nevertheless, it was necessary to administer this placement 
test to appropriately identify participants’ proficiency level. In addition to this, a 
background questionnaire was also administered in order to gather information 
regarding participants’ personal information (e.g. age, gender, mother tongue) and 
FL learning experience (adapted from Beltran-Palanques, 2013). Data gathered 
from the questionnaire revealed that participants were bilingual (i.e. Catalan and 
Spanish) and had studied English at school, secondary school and university. 

3.2. The speech act under investigation 

The pragmatic aspect under investigation in this study is the speech act of 
apologies. According to Searle (1979), apologies fall into the category of 
expressives, since they “express the psychological state specified in the sincerity 
condition about a state of affairs specified in the propositional content” (p. 15). 
Apologies are here defined as a “compensatory action to an offense in the doing 
of which S (the speaker) was causally involved and which is costly to H (the 
hearer)” (Bergman & Kasper, 1993, p. 82). Then, apologies can be used as 
remedial exchanges to restore harmony between speakers after a given offense 
(Goffman, 1971). 

People often take part in remedial actions in which they attempt to save face 
(Brown & Levinson, 1987) and restore the social harmony of the speech 
community (Goffman, 1971). Following Brown and Levinson (1987), an 
apology is typically viewed as negative politeness whose main goal is that of 
providing a redressive action. From the domain of politeness, an apology is seen 
as a communicative event in which the speaker (i.e. the apologiser) should take 
into account the other interlocutor’s face (i.e. the apologisee) in order to restore 
the situation (Brown & Levinson, 1987). Lienee, apologising, as mentioned by 
Bataineh and Bataineh (2006), is a face-saving act for the hearer and a face- 
threatening act for the speaker. 
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3.3. Instruments and procedure 

The research methodology employed in this study consisted of interactive written 
DCTs (Beltran-Palanques, 2013). This consists of eight different situations 
containing the following variables: social status (i.e. equal and hearer-dominant), 
social distance (i.e. stranger and acquaintance), and level of offense (high and 
low). However, in the present study, only four of the eight situations used in the 
aforementioned study were selected, specifically those involving the following 
variables: social status (i.e. equal), social distance (i.e. stranger and acquaintance), 
and level of offense (high and low). Table 1 shows the different situations. 


Table 1. Situations 


Context of the situation 

Variables 

Social status 

Social distance 

Level of offense 

Sit. 1 

Bookshop 

Equal 

Stranger 

Low 

Sit. 2 

University 

Equal 

Acquaintance 

High 

Sit. 3 

Student’s flat 

Equal 

Acquaintance 

Low 

Sit. 4 

University 

Equal 

Stranger 

High 


Participants were grouped in pairs and they were asked to complete the 
interactive written DCTs in pairs. Immediately after the completion of each 
written cognitive task, participants took part in the retrospective verbal report. 
Tasks were performed using Skype. However, due to the interactive nature 
of the written DCTs, only one participant of each pair elicited the speech act 
under investigation (i.e. apologies). Participants could read back their written 
production before engaging them in the verbal probes. In so doing, participants 
were exposed to their own production, in order to make them aware of what 
they produced. Participants were allowed to use Catalan, Spanish and/or English 
during the verbal probes, since in this case, the major goal was to examine 
participants’ thoughts while performing the tasks, rather than exploring their 
spoken competence in English. Furthermore, it was believed that allowing them 
to use their L 1 would facilitate the verbal reporting. Verbal reports were recorded 
in order to further examine participants’ contributions. The retrospective verbal 
reports were transcribed following Jefferson’s (2004) transcript notation. 
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4. Results and discussion 

This section provides an overview of the results of this study. Table 2 displays 
the results derived from the retrospective verbal reports. 


Table 2. Situations 


Verbal reports 

Situations 

Situation 1 

Situation 2 

Situation 3 

Situation 4 

Grammar and lexicon 

7 

6 

7 

9 

Social status 

7 

4 

5 

8 

Social distance 

8 

5 

5 

9 

Level of offense 

9 

12 

10 

15 


Each participant was asked to take part in the verbal report, which focused 
on aspects related to grammar and lexicon, and pragmatic knowledge. As 
mentioned above, only one of the participants in each pair took part in the 
apology production. Therefore, only the data derived from the participants who 
produced apology sequences are shown here. 

Regarding grammar and lexicon, retrospective verbal reports revealed that, in 
general, participants focused on aspects related to grammar and lexicon when 
planning their pragmatic production, especially in the fourth situation (i.e. 
nine participants out of 15). This could be related to the fact that this situation 
in particular could be more demanding for participants since it involved the 
following context: two equal participants who were strangers and whose level 
of offense was high. As a matter of fact, most participants indicated that this 
situation was very offensive for the other interlocutor since it involved damaging 
students’ class notes. It is worth mentioning that participants were students, so 
perhaps they perceived this as very offensive. 

Concerning social status, participants indicated that in situations 1 and 4 
this particular variable seemed to have affected the way they addressed their 
interlocutors. In this case, all the participants shared the same social status, and 
they revealed that the fact that they had the same status affected their production. 
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According to them, having a different social status could involve apologising in 
a different manner, thus, using more strategies to restore the situation. As regards 
this specific variable, participants did not provide much information, probably 
due to the fact that they all shared the same social status. 

Social distance, according to the results obtained in the verbal reporting, 
appeared not to have a great impact on participants’ pragmatic production. As a 
matter of fact, eight and nine participants mentioned this specific variable in the 
situations 1 and 4, in which social distance was that of stranger. According to the 
verbal reporting, in the other remaining two situations (i.e. 2 and 3), containing 
acquaintance social distance, a lower number of participants seemed to have 
paid attention to this issues, particularly five participants in each one. 

Finally, retrospective verbal reports revealed that the severity of offence played a 
paramount role. As a matter of fact, those situations whose level of offense was 
classified as high seemed to have received the attention of the majority of the 
participants. Particularly, situations 2 and 4, in which the severity of offense was 
classified as high, called participants’ attention as they involved situations which 
violated social norms. 

The results of this study are in line with previous works in the field, in which verbal 
reports were also instrumental in revealing participants’ information regarding 
their speech act performance in DCTs (Beltran-Palanques, 2013; Robinson, 1992; 
Woodfield, 2008, 2010). Robinson (1992) indicated that verbal reports were 
useful to obtain information about attended aspects, and indications of linguistic 
and pragmatic difficulties, among others. Woodfield (2008) found that verbal 
reports were instrumental in identifying participants’ attention while working on 
the tasks. Similarly, Woodfield’s (2010) study revealed that retrospective verbal 
reports provided information regarding participants’ cognitive processes while 
on task. Beltran-Palanques (2013) found that verbal reports were instrumental 
in providing information regarding participants’ pragmatic knowledge. In 
this explanatory study, retrospective verbal reports were also useful to obtain 
information related to participants’ attention to grammar and lexicon features as 
well as pragmatic knowledge. 
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5. Conclusions 

The aim of this explanatory study was to contribute to the growing body of 
literature that employs verbal reports in combination with other research 
instruments, particularly DCTs. Results from this study are consistent 
with previous research in the field, since retrospective verbal reports were 
instrumental in providing further information concerning participants’ speech 
act production. Several studies have concluded that verbal reports, in their 
various forms (i.e. concurrent and retrospective) can be of paramount interest, 
given that researchers can obtain further information related to learners’ 
pragmatic performance. This, in turn, can benefit language teachers in their tasks 
of developing instructional approaches aim to (i) integrate speech acts; and (ii) 
better understand how learners at different levels process and perform pragmatic 
utterances in contextualised situations. Empirical studies whose goal is to obtain 
participants’ thoughts while performing cognitive tasks, such as DCTs, should 
employ verbal reports, since this tool allows researchers to better understand 
participants’ pragmatic behaviour. Moreover, the use of DCTs and verbal reports 
can also be of paramount interest for instruction, as a tool to obtain information 
about learners’ pragmatic perfonnance, and to improve instructional approaches 
as well as design them drawing on empirical findings. 
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